Terminal, Operating Method Thereof, and Computer-Readable Recording Medium

ABSTRACT

Disclosed are a terminal, an operating method thereof, and a computer-readable recording medium. The operating method includes establishing a video call session between a first terminal and a second terminal, receiving, by the second terminal, data obtained by the first terminal, sequentially storing, by the second terminal, the data received from the first terminal in a buffer size, performing, by the second terminal, a validity check on the data stored in the buffer, and processing, by the second terminal, the data in response to a result of the validity check.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. Pat. Application No.17/161,893 filed Jan. 29, 2021, which claims priority to and the benefitof Korean Patent Application No. 2020-0012270 filed on Jan. 31, 2020,the disclosures of which are incorporated herein by reference in itsentirety.

BACKGROUND 1. Field of the Invention

The present invention relates to a terminal, an operating methodthereof, and a computer-readable recording medium and, morespecifically, to a terminal that allows a video call user not to beexposed to an unpleasant environment, an operating method thereof, and acomputer-readable recording medium.

2. Discussion of Related Art

With the development of communication technology and miniaturization ofelectronic devices, personal terminals are widely distributed to generalconsumers. In particular, recently, portable personal terminals such assmartphones or smart tablet computers are widely distributed. Mostterminals have communication functions. Users may search the Internet ortransmit or receive messages with other users using the terminals.

Further, with the development of small camera technology, smallmicrophone technology, small display technology, and small speakertechnology, most terminals, such as smartphones have a camera, amicrophone, a display, and a speaker. A user may record a voice orcapture a video including a voice using a terminal. The user may check arecorded voice through a speaker included in the terminal or check acaptured video through a display.

Further, the user may share the recorded voice or the captured videowith another user using a communication function of the terminal. Theuser may transmit a voice previously recorded or a video previouslycaptured to another user. Further, the user may transmit a voicecurrently being recorded by the terminal or a video currently beingcaptured to another user in real time.

Further, at the same time, another user may transmit a voice currentlybeing recorded by his or her own terminal or a video currently beingcaptured to the user in real time. The display included in the terminalof the user may simultaneously display the video currently beingcaptured by the terminal of the user and the video currently beingcaptured by the terminal of another user. Further, the speaker includedin the terminal of the user may simultaneously play both of the voicecurrently being recorded by the terminal of the user and a voicecurrently being recorded by a terminal of another user. In other words,the user and another user may make a video call with each other usingtheir own terminals.

When a user makes a video call with a counterpart, the user may beexposed to an inappropriate video or sound which is provided from thecounterpart. In particular, while a user makes a video call with acounterpart in a relationship in which they do not know each other,there may be a high possibility that the user is exposed to obscenevideos provided from the counterpart.

Various methods have been proposed to protect video call users fromharmful environments. However, there is a need for a method of reliablyblocking harmful elements while maintaining low latency in a process oftransmitting or receiving data in real time.

SUMMARY OF THE INVENTION

The present invention is directed to providing a terminal that allows avideo call user not to be exposed to a harmful environment, an operatingmethod thereof, and a computer-readable recording medium.

According to an aspect of the present invention, there is provided anoperating method of a terminal, which includes establishing a video callsession between a first terminal and a second terminal, receiving, bythe second terminal, data obtained by the first terminal, sequentiallystoring, by the second terminal, the data received from the firstterminal in a buffer size, performing, by the second terminal, avalidity check on the data stored in the buffer, and processing, by thesecond terminal, the data in response to a result of the validity check.

The method may further include decoding, by the second terminal, thedata, and in the decoding of the data, the data for which the validitycheck is completed in the performing of the validity check may bedecoded. Alternatively, the method may further include decoding, by thesecond terminal, the data, and in the performing of the validity check,the validity check may be performed on the data decoded in the decodingof the data stored in the buffer.

In the performing of the validity check, a machine learning model may beused to determine whether the data includes an improper element.

In the processing of the data, it may be determined not to play dataincluding an improper element as a result of the validity check.

In the processing of the data, it may be determined to play alternativedata for a period of time corresponding to the data including animproper element.

The data may include at least one of image data, audio data, and textdata.

The performing of the validity check may include performing the validitycheck on a first data set stored in the buffer, performing the validitycheck on a second data set stored in the buffer, and performing thevalidity check on a third data set stored in the buffer. The first tothird data sets may be data sets sequentially stored in the buffer.

As a result of performing the validity check, when it is determined thatthe first data set and the third data set do not include an improperelement and the second data set includes an improper element, in theprocessing of the data, it may be determined to play the first data setand the third data set and not to play the second data set.

In the processing of the data, image data included in the second dataset may be subjected to blurring or mosaic processing, audio dataincluded in the second data set may be subjected to mute processing, andtext data included in the second data set may not be displayed.

Meanwhile, there is provided a computer-readable recording medium onwhich a program for performing the operating method of the terminalaccording to the present invention is recorded.

According to still another aspect of the present invention, there isprovided a terminal including a communication interface configured toestablish a video call session between the terminal and another terminaland allow the terminal and another terminal to transmit or receive datato or from each other, a data storage unit configured to sequentiallystore data received from the another terminal through the video callsession according to a time at which the data is received, a validitycheck unit configured to perform a validity check on the data stored inthe data storage unit, and a control unit configured to process the datain response to a result of the validity check.

Further, the terminal may further include a decoder configured to decodethe data, and the decoder may decode the data for which the validitycheck is completed in the validity check unit.

Further, the validity check unit may use a machine learning model todetermine whether the data includes an improper element.

The control unit may determine not to play the data including animproper element as a result of the validity check.

The control unit may determine to play alternative data for a period oftime corresponding to the data including the improper element.

The data may include at least one of image data, audio data, and textdata.

The validity check unit may perform a validity check on a first dataset, a second data set, and a third data set which are stored in thedata storage unit, and the first to third data sets may be data setssequentially stored in the data storage unit.

As a result of performing the validity check, when it is determined thatthe first data set and the third data set do not include an improperelement and the second data set includes an improper element, thecontrol unit may determine to play the first data set and the third dataset and determines not to play the second data set.

The control unit may perform blurring or mosaic processing on image dataincluded in the second data set, perform mute processing on audio dataincluded in the second data set, and allow text data included in thesecond data set not to be displayed.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and advantages of the presentinvention will become more apparent to those of ordinary skill in theart by describing exemplary embodiments thereof in detail with referenceto the accompanying drawings, in which:

FIG. 1 is a schematic diagram illustrating an environment in whichterminals operate according to the present invention;

FIG. 2 is a flowchart schematically illustrating an operating method ofa terminal according to an embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating an operation in which data isstored in a buffer according to an embodiment of the present invention;

FIG. 4 is a flowchart schematically illustrating an operating method ofa terminal according to another embodiment of the present invention;

FIG. 5 is a schematic diagram illustrating an operating method of aterminal according to a related art; and

FIG. 6 is a schematic diagram illustrating a structure of a terminalaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Advantages and features of the present invention and methods ofachieving the same will be clearly understood with reference to theaccompanying drawings and embodiments described in detail below.However, the present invention is not limited to the embodiments to bedisclosed below but may be implemented in various different forms. Theembodiments are provided in order to fully explain the presentembodiments and fully explain the scope of the present embodiments forthose skilled in the art. The scope of the present embodiments is onlydefined by the appended claims. Like reference numerals refer to likeelements throughout the specification.

It will be understood that, although the terms “first,” “second,” etc.may be used herein to describe various elements, these elements shouldnot be limited by these terms. These terms are only used to distinguishone element from another. Therefore, a first element described below maybe a second element within the technological scope of the presentinvention.

Terms used in this specification are considered in a descriptive senseonly and not for purposes of limitation. In this specification, thesingular forms include the plural forms unless the context clearlyindicates otherwise. It will be understood that the terms “comprise” or“comprising,” when used herein, specify some stated components, steps,operations and/or elements but do not preclude the presence or additionof one or more other components and operations.

Unless otherwise defined, all terms (including technical and scientificterms) used herein can be used as is customary in the art to which thepresent invention belongs. Also, it will be further understood thatterms, such as those defined in commonly used dictionaries, will not beinterpreted in an idealized or overly formal sense unless expressly sodefined herein.

FIG. 1 is a schematic diagram illustrating an environment in whichterminals operate according to the present invention. Referring to FIG.1 , an environment, in which a first terminal 10 and a second terminal20 operate, may include a server 100, and the first terminal 10 and thesecond terminal 20 which are connected to the server 100. Descriptionsof terminals according to the present invention may be replaced withdescriptions of the first terminal 10 and the second terminal 20illustrated in FIG. 1 . For convenience of description, only twoterminals, that is, the first terminal 10 and the second terminal 20,are illustrated in FIG. 1 , but more than two terminals may be included.With respect to terminals that may be added, the descriptions of thefirst terminal 10 and the second terminal 20 may be applied, except fora description to be specifically described.

The server 100 may be connected to a communication network. The server100 may be connected to other external devices via the communicationnetwork. The server 100 may transmit or receive data to or from anotherdevice connected thereto.

The communication network connected to the server 100 may include awired communication network, a wireless communication network, or acomplex communication network. The communication network may include amobile communication network using third-generation (3G) communication,Long-Term Evolution (LTE) communication, LTE Advance (LTE-A)communication, or the like. The communication network may include awired or wireless communication network using Wi-Fi, Universal MobileTelecommunications System (UMTS) and/or General Packet Radio Service(GPRS), Ethernet, or the like. The communication network may include ashort-range communication network using magnetic secure transmission(MST), radio frequency identification (RFID), near field communication(NFC), Zigbee, Z-Wave, Bluetooth, Bluetooth Low Energy (BLE), infrared(IR) communication, or the like. The communication network may include alocal area network (LAN), a metropolitan area network (MAN), a wide areanetwork (WAN), or the like.

The server 100 may receive data from at least one of the first terminal10 and the second terminal 20. The server 100 may perform an operationusing the data received from at least one of the first terminal 10 andthe second terminal 20. The server 100 may transmit a result of theoperation to at least one of the first terminal 10 and the secondterminal 20.

The server 100 may receive a relay request from at least one of thefirst terminal 10 and the second terminal 20. The server 100 may selectthe terminal that transmits the relay request. For example, the server100 may select the first terminal 10 and the second terminal 20.

The server 100 may relay a communication connection between the selectedfirst and second terminals 10 and 20. For example, the server 100 mayrelay a video call connection between the first terminal 10 and thesecond terminal 20 or relay a transmission or reception connection fortext. The server 100 may transmit connection information about the firstterminal 10 to the second terminal 20 and transmit connectioninformation about the second terminal 20 to the first terminal 10.

The connection information about the first terminal 10 may include, forexample, an Internet protocol (IP) address and a port number of thefirst terminal 10. When the first terminal 10 receives the connectioninformation about the second terminal 20, the first terminal 10 mayattempt to connect to the second terminal 20 using the receivedconnection information.

When the connection attempt of the first terminal 10 to the secondterminal 20 or the connection attempt of the second terminal 20 to thefirst terminal 10 is successful, a video call session may be establishedbetween the first terminal 10 and the second terminal 20. The firstterminal 10 may transmit a video or a sound to the second terminal 20through the video call session. The first terminal 10 may encode thevideo or the sound as a digital signal and transmit the encoded resultto the second terminal 20.

Further, the first terminal 10 may receive a video or a sound from thesecond terminal 20 through the video call session. The first terminal 10may receive a video or a sound which is encoded as a digital signal anddecode the received video or sound.

The second terminal 20 may transmit the video or the sound to the firstterminal 10 through the video call session. Further, the second terminal20 may receive the video or the sound from the first terminal 10 throughthe video call session. Accordingly, a user of the first terminal 10 anda user of the second terminal 20 may make a video call with each other.

Each of the first terminal 10 and the second terminal 20 may include,for example, a desktop computer, a laptop computer, a smartphone, asmart tablet computer, a smart watch, a mobile terminal, a digitalcamera, a wearable device, a portable electronic device, or the like.The first terminal 10 and the second terminal 20 may execute programs orapplications. Each of the first terminal 10 and the second terminal 20may be the same type of device or may be different types of devices.

FIG. 2 is a flowchart schematically illustrating an operating method ofa terminal according to an embodiment of the present invention.

Referring to FIG. 2 , the operating method of the terminals according tothe embodiment of the present invention includes establishing a videocall session between a first terminal and a second terminal (S110),transmitting data to the second terminal (S120), storing received datain a buffer (S130), checking a validity of the data (S140), andprocessing the data (S150).

In operation S110, a video call session is established between twoterminals, that is, the first terminal 10 and the second terminal 20.The video call session may be established by the server 100 describedwith reference to FIG. 1 or may be actively established by the firstterminal 10 and the second terminal 20.

In operation S120, the data obtained by the first terminal 10 istransmitted to the second terminal 20. The first terminal 10 and thesecond terminal 20 transmit or receive the data to or from each otherthrough the video call session. The data may include at least one ofvideo, voice, and text. When data generated by the first terminal 10 istransmitted to the second terminal 20, the data generated by the firstterminal 10 may be encoded and transmitted to the second terminal 20 andthe encoded data may be decoded in the second terminal 20. When datagenerated by the second terminal 20 is transmitted to the first terminal10, the data encoded by the second terminal 20 may be decoded in thefirst terminal 10.

In operation S130, the second terminal 20 sequentially stores datareceived from the first terminal 10 in a buffer size. For example, whenthe size of the buffer is preset to n bytes, the data received from thefirst terminal 10 may be stored in the buffer of the second terminal 20in units of n bytes. When data having 10 X n bytes are transmitted fromthe first terminal 10 to the second terminal 20 during a specific timeinterval, the data may be sequentially stored in the buffer in units ofn bytes in the order in which the data are transmitted from the firstterminal 10. Meanwhile, here, the size of the buffer may refer to a sizeof the entire buffer but may also refer to a size of a unit bufferstoring received data.

Meanwhile, the size of the unit buffer may be determined to correspondto a predetermined period of time. For example, the size of the unitbuffer may be determined as a size of the data corresponding to 1 ms. Inthis case, in the unit buffer, the data corresponding to 1 ms may besequentially stored in the entire buffer in the order in which the dataare received.

In operation S140, the second terminal 20 performs a validity check ondata stored in the buffer. The validity check is to check whether thedata received from the first terminal 10 is valid and includes checkinga validity of the data itself and whether the data includes an improperelement.

For example, the data may include at least one of image data, audiodata, and text data. In operation S140, it may be checked whether theimage data, the audio data, or the text data includes an improperelement.

In an embodiment, in operation S140, it may be checked whether the imagedata included in the data includes an improper element corresponding toat least one of rude gestures, insulting gestures, racist gestures,sexual content, nudity, genitalia, sexual activity, unsanitary content,excreta, acts of excreting, antisocial content, misanthropic content,illegal activity, criminal activity, hate crime, acts of violence, actsof abuse, acts of self-harm, weapons, drugs, antisocial symbols, hatefulcontent, threatening content, phobia-inducing content, and blood.

Further, in operation S140, it may be checked whether the audio data ortext data included in the data includes an improper elementcorresponding to at least one of abusive, intimidatory, hateful,anti-social, misanthropic, sexual, criminal, and offensive statements.Alternatively, it may be checked whether the audio data included in thedata includes at least one improper element among groaning, a gaspingsound, screaming, and shrieking. As another example, it may be checkedwhether the audio data included in the data includes at least oneimproper element among a roaring sound, a gunfire sound, a vomitingsound, a hitting sound, a whipping sound, a knife sound, a swingingsound, a stabbing sound, a sawing sound, a grating sound, a creakingsound, a sound having a pitch greater than or equal to a specificthreshold value, and a sound having a loudness greater than or equal toa specific threshold value.

Meanwhile, in another embodiment, in operation S140, a machine learningmodel may be used to determine whether the data includes an improperelement. The machine learning model may be a neural network including atleast two layers. The machine learning model may include an input layerand an output layer. The machine learning model may further include atleast one hidden layer.

The machine learning model may be a classification model trained usingat least one video, voice, or text and information on whether the atleast one video, voice, or text includes the improper element. Themachine learning model may use at least one feature point included in anarbitrary video, voice, or text as an input value. The machine learningmodel may predict whether the at least one feature point corresponds tothe improper element.

The machine learning model may be trained using a deep learningalgorithm. The machine learning model may include at least one oftechniques of a deep neural network (DNN), a convolutional neuralnetwork (CNN), a recurrent neural network (RNN), a region-basedconvolutional neural network (R-CNN), a restricted Boltzmann machine(RBM), a deep belief network (DBN), and a deep Q-network.

The machine learning model may include a CNN. The machine learning modelmay include AlexNet, ZFNet, GoogLeNet, VGGNet, ResNet, Inception-ResNet,Inception-v2, Inception-v3, or Inception- v4.

In an embodiment, a subject that determines whether an improper elementis included in the data received using the machine learning model may bethe server 100 or the first terminal 10 or the second terminal 20 thatreceives the data. That is, the receiving terminal may determine whetherthe improper element is included in the data using the machine learningmodel.

Meanwhile, in another embodiment of the present invention, operationS140 may include performing a validity check on a first data set storedin the buffer, performing a validity check on a second data set storedin the buffer, and performing a validity check on a third data setstored in the buffer. Here, the first to third data sets may be datasets sequentially stored in the buffer.

Therefore, it may be understood that, after the video call session isestablished in operation S110, operations S120 to S150 are repeatedlyperformed on the data sets stored in the unit buffer.

When the video call session is established and the data transmission isconsecutively performed between the first terminal 10 and the secondterminal 20 in real time, the validity check may be consecutivelyperformed on the data sets sequentially stored in the buffer. The dataset for which the validity check is completed may proceed to operationS150, and the validity check may be performed on the data set storedafter the corresponding data set.

In operation S150, the data is processed in response to a result of thevalidity check. In an embodiment, in operation S150, when it isdetermined that the data includes an improper element as a result of thevalidity check in operation S140, it may be determined not to play thecorresponding data in the second terminal 20.

For example, the data sets sequentially stored in the buffer may bechecked in the order in which the data are received, and it may bedetermined not to play the data including the improper element. When itis determined that the image data includes an improper element, thecorresponding data may not be played through a display of the secondterminal 20. When it is determined that the audio data includes animproper element, the corresponding data may not be played through aspeaker of the second terminal 20.

When it is determined that n^(th) stored data (n^(th) data) includes animproper element, the n^(th) data may not be played and a validity checkmay be performed on (n+1 )^(th) stored data ((n+ 1)^(th) data). When itis determined that the (n+ 1)^(th) data does not include an improperelement, the (n+ 1)^(th) data may be normally played in the secondterminal 20 and a validity check may be performed on (n+2)^(th) storeddata ((n+2)^(th) data). Whether the (n+2)^(th) data is played may bedetermined in operation S150 according to the result of the validitycheck in operation S140.

Meanwhile, in operation S150, it may be determined to play alternativedata for a period of time corresponding to the data including theimproper element. For example, when it is determined that the image dataincludes an improper element, it may be determined to play the imagedata included in the previous data set or to play an image obtained byperforming blurring or mosaic processing on the image data including theimproper element as an alternative image. When it is determined that theaudio data includes an improper element, the audio data included in theprevious data set may be played or the audio data including the improperelement may be muted. Alternatively, when it is determined that the textdata includes an improper element, it may be determined not to play thecorresponding text data or to display a message indicating that theimproper element is included.

FIG. 3 is a schematic diagram illustrating an operation in which data isstored in a buffer according to an embodiment of the present invention.

Referring to FIG. 3 , data transmitted from a terminal of a counterpartconnected through a video call session may be sequentially stored in abuffer of size m. In this case, the size m of the buffer may correspondto bytes of a predetermined size or to a predetermined time.

In FIG. 3 , it may be understood that a (n-1)^(th) data set (data setn-1), a n^(th) data set (data set n), and a (n+1)^(th) data set (dataset n+1) are sequentially transmitted from the terminal of thecounterpart.

A validity check may be sequentially performed on the (n-1)^(th) dataset (data set n-1) to the (n+ 1)^(th) data set (data set n+1) stored inthe buffer. The validity check corresponds to operation S140 describedwith reference to FIG. 2 , and a data processing operation correspondingto operation S150 is performed on the data sets for which the validitycheck is completed.

When the buffer illustrated in FIG. 3 is the component included in thesecond terminal 20 described with reference to FIGS. 1 and 2 , it may beunderstood that the (n-1)^(th) data set (data set n-1) to the (n+1)^(th)data set (data set n+1) are sequentially transmitted from the firstterminal 10.

It may be understood that sizes of the (n-1)^(th) data set (data setn-1) to the (n+1)^(th) data set (data set n+1) are identical to eachother. Alternatively, it may be understood that play times of the(n-1)^(th) data set (data set n-1) to the (n+1)^(th) data set (data setn+1) are also identical to each other.

That is, a space in which the (n-1)^(th) data set (data set n-1) to the(n+1)^(th) data set (data set n+1) are stored may be defined as eachunit buffer included in the buffer, and data processing may besequentially performed in a first in first out (FIFO) manner. Thevalidity check and processing according to a result of the validitycheck may be sequentially performed on the data (or the data sets)stored in the buffer.

FIG. 4 is a flowchart schematically illustrating an operating method ofa terminal according to another embodiment of the present invention.

Referring to FIG. 4 , the operating method of the terminal according toanother embodiment of the present invention includes establishing avideo call session between a first terminal and a second terminal(S210), transmitting data to the second terminal (S220), storingreceived data in a buffer (S230), performing a validity check on thedata (S240), decoding the data (S250), and processing the data (S260).In the establishing of the video call session between the first terminaland the second terminal (S210), the transmitting of the data to thesecond terminal (S220), the storing of the received data in the buffer(S230), the performing of the validity check on the data (S240), and theprocessing of the data (S260), substantially the same operations asoperations S110 to S150 described with reference to FIG. 2 areperformed, and thus detailed descriptions thereof will not be repeated.

The data transmitted from the first terminal 10 is encoded as digitaldata and transmitted to the second terminal 20 in a compressed form. Inoperation S250, an operation of decoding the encoded data is performed.In operation S250, data decoding is performed on the data for which thevalidity check is completed, and in operation S260, data processing isperformed in response to a result of the validity check. The data thatdoes not go through the validity check in operation S240, that is, thedata including the improper element, may not be decoded in operationS250.

In another embodiment, the validity check in operation S240 may beperformed after the data is decoded. Accordingly, after the encoded datatransmitted to the second terminal 20 is decoded, a validity check maybe performed. The data for which the validity check is completed may beprocessed in operation S260 in response to a result of the validitycheck. After the data is decoded, the data that does not go through thevalidity check in operation S240, that is, the data including theimproper element, may be processed in operation S260 so that theimproper element is blocked for the user, such as by determining not toplay the data or playing alternative data for a period of timecorresponding to the data.

The second terminal 20 of the embodiment may sequentially store imagedata sets received from the first terminal 10 in the buffer and mayperform a validity check on each data set according to the stored order.Meanwhile, in an embodiment, the second terminal 20 may decode thereceived image data sets and sequentially store the decoded image datasets in the buffer. However, it is possible to prevent inappropriatedata from being played by determining whether to play the data accordingto the result of the validity check without determining whether to playthe data immediately after the decoding. The second terminal 20 maydetermine whether to play the data sets according to the result of thevalidity check. In this way, it is possible to provide improved userconvenience without the intervention of the server by performing avalidity check for checking inappropriate elements of the received datasets using the machine learning model in each terminal.

Like a structure of the buffer of FIG. 3 , the data decoding inoperation S250 is performed in the order of being stored in the buffer,and when decoding for one piece of data (or one data set) is completed,the decoding is performed on data (or a data set) stored in the bufferafter the decoding is performed on the corresponding data.

FIG. 5 is a schematic diagram illustrating an operating method of aterminal according to a related art.

Referring to FIG. 5 , in the terminal according to the related art,decoding and a validity check are simultaneously performed on receiveddata. Since the decoding and the validity check are simultaneouslyperformed on the same data, even when the data including an improperelement is found, the decoded data may be transmitted to an input andoutput interface and displayed on the terminal of a user. Therefore,there is a problem in that even when playing of the data including theimproper element stops, some pieces of corresponding data are inevitablyexposed to the user.

In the terminal according to the related art, by simultaneouslyperforming the decoding and the validity check, latency associated withdata communication may be reduced but such a method is not desirable interms of user health.

On the other hand, according to the operating method of the terminalaccording to the embodiment of the present invention, the decoding maynot be performed on the data that does not go through the validitycheck, and even when the data is transmitted to the input and outputinterface, the data may not be displayed normally to the user.Therefore, it is possible to expect an effect of fundamentally blockingimproper elements that may harm the user experience.

FIG. 6 is a schematic diagram illustrating a structure of a terminalaccording to an embodiment of the present invention.

Referring to FIG. 6 , a terminal 30 according to the embodiment of thepresent invention includes a communication interface 31, a data storageunit 32, a validity check unit 33, and a control unit 34. Each of thefirst terminal 10 and the second terminal 20 of FIG. 1 may includesubstantially the same components as the terminal 30 illustrated in FIG.6 . The control unit 34 includes at least one processor. Thecommunication interface 31 includes at least one transceiver. The datastorage unit 32 includes at least one memory. The validity check unit 33includes at least one processor. Here, the at least one processor may beembodied as a various number of hardware, software and/or firmwarestructures that execute corresponding functions described herein.

The communication interface 31 may establish a video call sessionbetween the terminal 30 and another terminal and allow the terminal 30to transmit or receive data to or from the another terminal. In theterminal 30, an application for executing a video call may be installedand the terminal 30 may make a video call with another terminal on whichthe same application is installed. The communication interface 31 maytransmit a video call request from the terminal 30 to a terminal of acounterpart or receive a video call request from the terminal of thecounterpart. In addition, the communication interface 31 may transmitinformation about the terminal 30 to the terminal of the counterpart orreceive information about the terminal of the counterpart and mayestablish a video call session using the information about theterminals.

When the video call session is established, the communication interface31 may transmit data obtained by the terminal 30 to the anotherterminal. The data may include at least one of video, a voice, and text.When the data generated by the terminal 30 is transmitted to the anotherterminal, the data generated by the terminal 30 may be encoded andtransmitted to the another terminal and the encoded data may be decodedin the another terminal. When data generated by the another terminal istransmitted to the terminal 30, the data encoded by the another terminalmay be decoded in the terminal 30. Therefore, the terminal 30 mayfurther include a decoder for decoding encoded data. In addition, thedecoder may perform decoding on the data for which a validity check iscompleted by the validity check unit 33. Alternatively, the decoder maytransmit the decoded data to the validity check unit 33 to determinewhether an improper element is included in the decoded data.

The data storage unit 32 sequentially stores data received from theanother terminal in a buffer having a preset size. For example, when thesize of the buffer is preset to n bytes, the data received from theanother terminal may be stored in the buffer of the data storage unit 32in units of n bytes. When the data having 10 X n bytes are transmittedfrom the another terminal to the terminal 30 during a specific timeinterval, the data may be sequentially stored in the buffer in units ofn bytes in the order in which the data are transmitted from the anotherterminal. Meanwhile, here, the size of the buffer may refer to a size ofthe entire buffer but may also refer to a size of a unit buffer storingreceived data.

Meanwhile, the size of the unit buffer may be determined to correspondto a predetermined period of time. For example, the size of the unitbuffer may be determined as a size of data corresponding to 1 ms. Inthis case, in the unit buffer, the data corresponding to 1 ms may besequentially stored in the entire buffer in the order in which the dataare received.

The validity check unit 33 performs a validity check on the data storedin the data storage unit 32. The validity check is to check whether thedata received from the another terminal is valid and includes checking avalidity of the data itself and whether the data includes an improperelement.

For example, the data may include at least one of image data, audiodata, and text data, and the validity check unit 33 may check whetherthe image data, the audio data, or the text data includes an improperelement.

In an embodiment, the validity check unit 33 may check whether the imagedata included in the data includes an improper element corresponding toat least one of rude gestures, insulting gestures, racist gestures,sexual content, nudity, genitalia, sexual activity, unsanitary content,excreta, acts of excreting, antisocial content, misanthropic content,illegal activity, criminal activity, hate crime, acts of violence, actsof abuse, acts of self-harm, weapons, drugs, antisocial symbols, hatefulcontent, threatening content, phobia-inducing content, and blood.

Further, the validity check unit 33 may check whether the audio data ortext data included in the data includes an improper elementcorresponding to at least one of abusive, intimidatory, hateful,anti-social, misanthropic, sexual, criminal, and offensive statements.Alternatively, it may be checked whether the audio data included in thedata includes at least one improper element among groaning, a gaspingsound, screaming, and shrieking. As another example, it may be checkedwhether the audio data included in the data includes at least oneimproper element among a roaring sound, a gunfire sound, a vomitingsound, a hitting sound, a whipping sound, a knife sound, a swingingsound, a stabbing sound, a sawing sound, a grating sound, a creakingsound, a sound having a pitch greater than or equal to a specificthreshold value, and a sound having a loudness greater than or equal toa specific threshold value.

Meanwhile, in another embodiment, the validity check unit 33 may use amachine learning model to determine whether the data includes animproper element. The machine learning model may be a neural networkincluding at least two layers. The machine learning model may include aninput layer and an output layer. The machine learning model may furtherinclude at least one hidden layer.

The machine learning model may be a classification model trained usingat least one video, voice, or text and information on whether the atleast one video, voice, or text includes the improper element. Themachine learning model may use at least one feature point included in anarbitrary video, voice, or the text as an input value. The machinelearning model may predict whether the at least one feature pointcorresponds to the improper element.

The machine learning model may be trained using a deep learningalgorithm. The machine learning model may include at least one of a DNN,a CNN, an RNN, an R-CNN, an RBM, a DBN, and a deep Q-network.

The machine learning model may be a CNN. The machine learning model mayinclude AlexNet, ZFNet, GoogLeNet, VGGNet, ResNet, Inception-ResNet,Inception-v2, Inception-v3, or Inception- v4.

In this way, the validity check unit 33 may perform machine learning onthe data received by the terminal 30 so that the terminal 30 itself,which is not the server, may determine whether an improper element isincluded in the data. Since the terminal 30 receiving the data performsa validity check, a result of the machine learning may vary for eachterminal 30, which may be advantageous in providing a user-customizedresult.

Meanwhile, in another embodiment of the present invention, the validitycheck unit 33 may perform a validity check on a first data set to athird data set stored in the data storage unit 32. In this case, thefirst to third data sets refer to data sets sequentially stored in thedata storage unit 32.

Therefore, after the video call session is established, the validitycheck is performed on a data set stored first in time in the datastorage unit 32 in the order of being stored and the validity check isperformed on the data sets stored thereafter.

When the video call session is established and the data transmission isconsecutively performed between the terminal 30 and the another terminalin real time, the validity check may be consecutively performed on thedata sets sequentially stored in the data storage unit 32. The data setfor which the validity check is completed may undergo a data processingprocess, and the validity check may be performed on the data set storedafter the corresponding data set.

In an embodiment, the validity check unit 33 may perform a validitycheck on the decoded data. For example, the data storage unit 32 maysequentially store encoded data received from another terminal, thedecoder may sequentially decode the stored data, and the validity checkunit 33 may perform a validity check on the decoded data. Alternatively,the data storage unit 32 may sequentially store decoded data, and thevalidity check unit 33 may perform a validity check on the decoded datasequentially stored in the data storage unit 32.

The control unit 34 processes the data in response to a result of thevalidity check. In an embodiment, when it is determined that the dataincludes an improper element as a result of the validity check in thevalidity check unit 33, the control unit 34 may determine not to playthe corresponding data in the terminal 30.

For example, the data sets sequentially stored in the data storage unit32 may be checked in the order in which the data are received, and itmay be determined not to play the data including the improper element.When it is determined that the image data includes an improper element,the corresponding data may not be played through a display of theterminal 30. When it is determined that the audio data includes animproper element, the corresponding data may not be played through aspeaker of the terminal 30. The display and the speaker may constitutean input and output interface (not illustrated) included in the terminal30.

When it is determined that n^(th) stored data (n^(th) data) includes animproper element, the n^(th) data may not be played and a validity checkmay be performed on (n+1)^(th) stored data ((n+1)^(lh) data). When it isdetermined that the (n+1)^(th) data does not include an improperelement, the (n+1)^(th) data may be normally played in the terminal 30and a validity check may be performed on (n+2)^(th) stored data((n+2)^(th) data). Whether the (n+2)^(th) data is played may bedetermined in the control unit 34 according to a result of the validitycheck.

Meanwhile, the control unit 34 may determine to play alternative datafor a period of time corresponding to the data including the improperelement. For example, when it is determined that the image data includesan improper element, it may be determined to play the image dataincluded in the previous data set or to play an image obtained byperforming blurring or mosaic processing on the image data including theimproper element as an alternative image. When it is determined that theaudio data includes an improper element, the audio data included in theprevious data set may be played or the audio data including the improperelement may be muted. Alternatively, when it is determined that the textdata includes an improper element, it may be determined not to play thecorresponding text data or to display a message indicating that theimproper element is included.

The embodiments described above may also be realized in the form of arecording medium including instructions executable by a computer, suchas a program module executed by a computer. Computer-readable media maybe any available media which may be accessed by the computer and includeany one of volatile and non-volatile media and separable andnon-separable media.

Further, the computer-readable media may include computer storage media.The computer storage media may include any one of volatile andnon-volatile media and separable and non-separable media, which areimplemented using any method or technology for the storage ofinformation such as computer-readable instructions, data structures,program modules, or other pieces of data.

According to the present invention, a terminal that allows a video calluser not to be exposed to a harmful environment, an operating methodthereof, and a computer-readable recording medium can be provided.

The embodiments of the present invention have been described above withreference to the accompanying drawings. However, it should be understoodby those skilled in the art that various modifications can be madewithout departing from the scope of the present invention and withoutchanging essential features thereof. Therefore, the above-describedembodiments should be considered in a descriptive sense only and not forpurposes of limitation.

What is claimed is:
 1. An operating method of a terminal, the methodcomprising: establishing a video call session between a first terminaland a second terminal; receiving, by the second terminal, a plurality ofsets of data obtained at the first terminal; sequentially storing, bythe second terminal, subsets of the plurality of sets of data receivedfrom the first terminal in a buffer; performing, by the second terminal,a validity check on the data stored in the buffer; and processing, bythe second terminal, the data stored in the buffer in response to aresult of the validity check.
 2. The method of claim 1, wherein theperforming of the validity check includes decoding the data stored inthe buffer and performing the validity check on the data.
 3. The methodof claim 1, wherein the sequentially storing of the data in the bufferincludes decoding the data received from the first terminal andsequentially storing the data in the buffer.
 4. The method of claim 1,wherein the sequentially storing of the data in the buffer includesstoring the data sequentially in a size of a unit buffer, wherein thesize of the unit buffer is determined as a size of a data correspondingto a unit of video playback time.
 5. The method of claim 1, wherein, inthe performing of the validity check, a machine learning model is usedto determine whether the data includes an improper element.
 6. Themethod of claim 1, wherein the processing of the data comprises,determining not to play data including an improper element as a resultof the validity check.
 7. The method of claim 1, wherein the processingof the data comprises, determining to play alternative data for a periodof time corresponding to the data including an improper element.
 8. Themethod of claim 1, wherein the performing of the validity checkincludes: performing the validity check on a first data set stored inthe buffer; performing the validity check on a second data set stored inthe buffer; and performing the validity check on a third data set storedin the buffer, and the first to third data sets are data setssequentially stored in the buffer.
 9. The method of claim 8, wherein, asa result of the performing of the validity check, when it is determinedthat the first data set and the third data set do not include animproper element and the second data set includes an improper element,the processing of the data comprises, determining to play the first dataset and the third data set and not to play the second data set.
 10. Themethod of claim 9, wherein, in the processing of the data, image dataincluded in the second data set is subjected to blurring or mosaicprocessing, audio data included in the second data set is subjected tomute processing, and text data included in the second data set is notdisplayed.
 11. A computer-readable recording medium on which a programfor performing the method according to claim 1 is recorded.
 12. Aterminal comprising: a transceiver configured to establish a video callsession between the terminal and another terminal and allow the terminaland another terminal to transmit or receive a plurality of sets of datato or from each other; a buffer configured to sequentially store subsetsof the plurality of sets of data received from the another terminalthrough the video call session according to a time at which the data isreceived; a validity check processor configured to perform a validitycheck on the data stored in the buffer; and a control processorconfigured to process the data stored in the buffer in response to aresult of the validity check.
 13. The terminal of claim 12, furthercomprising a decoder configured to decode the data, wherein the decoderdecodes the data for which the validity check is completed in thevalidity check processor.
 14. The terminal of claim 12, wherein thebuffer stores the data sequentially in a size of a unit buffer, whereinthe size of the unit buffer is determined as a size of a datacorresponding to a unit of video playback time.
 15. The terminal ofclaim 12, wherein the validity check processor uses a machine learningmodel to determine whether the data includes an improper element. 16.The terminal of claim 12, wherein, as a result of the validity check,the control processor determines not to play the data including animproper element.
 17. The terminal of claim 15, wherein the controlprocessor determines to play alternative data for a period of timecorresponding to the data including the improper element.
 18. Theterminal of claim 12, wherein the validity check processor performs avalidity check on a first data set, a second data set, and a third dataset which are stored in the buffer, and the first to third data sets aredata sets sequentially stored in the buffer.
 19. The terminal of claim18, wherein, as a result of performing the validity check, when it isdetermined that the first data set and the third data set do not includean improper element and the second data set includes an improperelement, the control processor determines to play the first data set andthe third data set and determines not to play the second data set. 20.The terminal of claim 19, wherein the control processor performsblurring or mosaic processing on image data included in the second dataset, performs mute processing on audio data included in the second dataset, and allows text data included in the second data set not to bedisplayed.